Some Applications of Logic to Feasibility in 
Higher Types 

Aleksandar Ignjatovic and Arun Sharma 
February 1, 2008 

Abstract 

While it is commonly accepted that computability on a Turing ma- 
chine in polynomial time represents a correct formalisation of the notion 
of a feasibly computable function, there is no similar agreement on how 
to extend this notion on Junctionals, i.e., what functionals should be con- 
sidered feasible. One possible paradigm was introduced by K. Mehlhorn 
in who extended Cobham's definition of feasible functions to type 
2 functionals. Subsequently, this class of functionals (with inessential 
changes of the definition) was studied by Townsand (|l3|) who calls this 
class POLY, and by Cook and Kapron ([^|) who call the same class ba- 
sic feasible functionals. Cook and Kapron gave an oracle Turing machine 
model characterization of this class. In this paper we demonstrate that 
the class of basic feasible functionals has recursion theoretic properties 
which naturally generalize the corresponding properties of the class of 
feasible functions, thus giving furter evidence that the notion of feasibil- 
ity of functionals mentioned above is correctly chosen. We also improve 
the Kapron-Cook result on machine representation. 

Our proofs are based on essential applications of logic. We introduce 
a weak fragment of second order arithmetic with second order variables 
ranging over functions from N N which suitably characterizes basic feasible 
functionals, and show that it is a useful tool for investigating the properties 
of basic feasible functionals. In particular we provide an example how one 
can extract feasible "programs" from mathematical proofs which use non- 
feasible functionals. 



1 Introduction 

1.1 Recursion Theory of Feasible Functionals 

In [|J Cobham has established the following fundamental result: functions com- 
putable on a Turing machine in polynomial time are exactly functions which 
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can be obtained from the basic functions^] o(x) = 0,sq(x) = 2x,si(x) = 
2x + l,i%(xx, . . . x n ) = Xk and x#y by composition and by limited recursion 
on notation: 

f(x,0) = g(x) (1) 

f(x,2y) = hi(x,y,f(x,y)), y>0 (2) 

f(x,2y+l) = h 2 (x,y,f(x,y)) (3) 

\f(x,y)\ < \k(x,y)\ (4) 

It is easy to see that the same class of functions is obtained if we replace the 
schema of limited recursion on notation by a schema which instead of (|J) and 
(| has 

f(x >y ) = h(x,yj(x,[%])), y>0 (5) 
Also, instead of condition we can take the condition 

\f(x,y)\<q(\x\,\y\) (6) 

where q is a polynomial with natural coefficients. 

Finally, the same set can be obtained if the schema of limited recursion 
on notation is replaced by the following version of the schema of primitive 
recursion, in which p and q are polynomials and /* is an auxiliary primitive 
recursive function. 



nm=9{y) (7) 

f*(y,x + l) = h(y,x,r(y,x)) (8) 
(Vx<p(\y\))(\f*(y,x)\<q(\y\)) (9) 
f(y) = r(y,p(\y\)) (10) 
From the foundational point of view we note that conditions (H) and (m 



limit the growth rate of the function being defined using functions which are 
particularly simple to compute (polynomials), in the sense that their definition 
does not involve any recursion. 

Definition 1 A function F : (N N fxfl'^N is called a type 2 functional of 
rank (fc, I). 

One of our aims is to find analogues of the above schemata for a class of 
functionals which can be seen as natural extension of the class of feasible func- 
tions. 

Definition 2 A functional of rank (k, I) is obtained by 

1 Recall that \x\ denotes the length of the binary representation of x, that is to say \(log2x) + 
1], with 1 1 = 0; denotes the greatest integer less than or equal to x/2; x#y is equal to 

2M - lwl. 
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Functional composition from H(f, y, x), Gi(/, x), . . . , Gi(f, x) if 



F(f, x) = H(f, Gi (/, x),...,Gi (/, £), x) 
Expansion from G(f, x) if 

F(f,g,x,ff)=G(f,x) 

Functional substitution from H{g, f, x), G\(f ', x, y), . . . , Gi(f, x, y) if 
F(f, x) = H(Xy.Gx(f, x,y),..., Xy.G t (f, x, y), f, x) 

Limited recursion on notation from G, Hi,H2,K if the following hold for all 
f e (N N ) k andallxeN 1 : 

F(f,S,0) = G(f,x) (11) 

F(f,x,2y) = H^fa^Ftf&y)), y>0 (12) 

F(/>,2y + l) = H 2 (lx,y,F{f,x,y)), (13) 

\F(f,x,y)\ < \K(f,x,y)\ (14) 

The last condition is equivalent to the condition F(f,x,y) < K*(f,x,y) for 
K* = (l#K(f,x,y))^l. 

Townsend (p3[) considered the least class of functionals which contains the 
polynomial time computable functions, the application functional Ap defined 
by Ap(f, x) = f(x), which is closed under expansion, functional composition, 
functional substitution and limited recursion on notation. He has also shown 
that the scheme of functional substitution is redundant; thus we introduce the 
following appropriately modified definition. 

Definition 3 The class B of basic feasible functionals is the least class of func- 
tionals which contains initial functions o(x) — 0, s (x) — 2x, si(x) = 2x + 1, 
ik(xi, . . . x n ) — Xk, £#y and the application functional Ap(f, x) = f[x), which 
is closed under expansion, functional composition and limited recursion on no- 
tation. 

The class of basic feasible functionals extends the class of feasible functions in 
a "minimal way" ; it is obtained using essentially the same closure conditions as 
the class of feasible functions (taking into account that the schema of functional 
substitution is redundant and so it can be omitted). The only difference is 
addition of the application functional to the set of basic functions. However, 
in any reasonable model of feasibility in higher types such functional must be 
considered feasible. Also, it is easy to see that the only functions which belong 
to the class of the basic feasible functionals are in fact just feasible functions. 
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As before, recursion on notation of type ( |12| ) together with ([13]) can be 
replaced with a single condition 

F(f,x,y) = H(f,x,y,F(f,x, L±yJ)). 

However, the growth rate of a basic feasible functional clearly cannot be 
majorized by a first order polynomial. Thus, we cannot expect to have simple 
analogues for schemas involving polynomial bounds or limited primitive recur- 
sion. The same applies to the machine models of feasibility in higher types. 
This is why we need the following definitions introduced by Kapron in Q . 

Definition 4 The functional of type N N ->■ N N defined such that f i-> |/j 
where 

l/IO) = m ax|/(y)| 

\y\<x 

is called the norm functional; the function \ f\ is called the norm of the function 
/• 

Kapron has shown that the functional of type (1,1) such that (/, x) i— ► |/| (|x|) 
is not basic feasible and that, in fact, it cannot be majorized by any basic feasible 
functional. 

The class of second-order polynomials was also introduced in 0. Second 
order polynomials play quite the same role which first order polynomials play 
for feasible functions of type N fe — ► N. However, there is a major difference 
in their nature which greatly complicates their applications: while first order 
polynomials are themselves feasible functions, the above result of Kapron implies 
that the second order polynomials are not feasible functionals. 

Definition 5 Let xq,x%,... and /o,/i,--- he sets of first and second order 
variables respectively; then the set of second order polynomials in |/o|, ■ ■ ■ 
and |a;o|, ■ ■ ■ JS defined inductively as the least set of terms of the language 
L p containing constants n for each natural number n and all terms \xq \, \xi \, . . . 
and which satisfies the following closure condition: if P. Q are second-order 
polynomials and fi is a second-order variable, then P + Q, P Q and \fi\(P) are 
also second-order polynomials. 

Clearly, second order polynomials take into account the growth rate of functions. 
However, they cannot be considered to be feasible: to compute |/|(|x|), we must 
compute |/(y)| for all y such that \y\ < \x\, which is exponentially many values 
of y (in \x\). Kapron's result showing that the norm is not basic feasible implies 
that the same holds of second order polynomials. 

We can now state the main results of this paper, which are the best possible 
analogues of the corresponding first order theorems. Due to the mentioned 
difficulty (non- feasibility of the second order polynomials), the proofs of these 
analogues use formal logic as an essential tool. 
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Definition 6 Let Q(\f\, \x\) be a second order polynomial and let G(f,x) and 
H(f, x, z, y) be two basic feasible functionals. Assume that the functional 
F(f, x, y) satisfies 



F(f,x,0) = G(f,x) 
F{f,x,y) = H{f,x,F{f,x,\_\y\),y), 
\F{f,S,y)\<Q{\fl\x\M)) 



(15) 



Then we say that F is defined from functionals G, H by polynomially bounded 
recursion on notation with the bound Q. 

Theorem: Assume that the functional F(f, x, y) is defined from the functionals 
G(f,x) and H(f,x,z,y) by polynomially bounded recursion on notation with 
the bound Q(\f\, \x\, \y\). Then F(f, x, y) is a basic feasible functional. 

Definition 7 Let P(\f\, |a?|) and Q(\f\, \x\) be two second order polynomials 
and let G(f, x) and H(f, x, y, z) be two basic feasible functionals. Assume that 
the functional F* (/, x, y) satisfies 



F*(f,x,0) = G(f,x); 
F*(f,x,y + 1) = H(f, x, F*(f, x, y),y); 

(yy<P(\f\,\x\))(\F*(f,x,y)\<Q(\f\,\x\)); 
(Vy)(y > P(\f\, \x\) - F*(f, x, y) = F*(f, x, P(\f\, \x\)); 
(V/)(V/*)((Vy < P(\f\, \S\))(F(f, x, y) - F(f*,x, y)) 
(Vy)(F(f,x,y) = F(f*,x,y))y, 



Then we say that F(f,x) is dehned from functionals G,H by polynomially 
bounded recursion of polynomial length with bounds (Q,P). 

Theorem: Assume that the functional F(f, x) is defined from the functionals 
G and H by polynomially bounded recursion of polynomial length with bounds 
(Q, P). Then the functional F(f, x) is a basic feasible functional. 



1.2 Turing Machine Characterization 

We use the usual model for computability with oracle Turing machines (OTM). 
Function inputs are presented using oracles, which are queried using separate 
write-only oracle input tapes and read-only oracle output tapes, while the ma- 
chine is in the oracle query state corresponding to the input function which is 



and that the functional F(f,x) is defined by 



F(f,x)=F*(f,x,P(\f\,\x\)). 



(16) 
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queried. To query function input / at the value x, x is written in binary nota- 
tion on the oracle input tape associated with /, and the corresponding oracle 
query state is entered. After entering the oracle state which corresponds to /, 
the value /(x) appears on the oracle output tape associated with /, the oracle 
input tape is then erased and both the writing head of the oracle input tape and 
the reading head of the oracle output tape are placed at the corresponding ini- 
tial cells of the tapes. Thus, iterations of the form /(/(. . . /(x) . . .)) cannot be 
computed without the machine having to copy the intermediate results from the 
oracle output tape to the oracle input tape. In general, there are two possible 
conventions for accounting for the running time of an oracle call. In Mehlhorn's 
model, an oracle call has unit cost, while in the Kapron/Cook model, the oracle 
call described above has a cost of |/(x)| time steps. Mehlhorn || and Cook and 
Kapron || proved the following theorems. 



Theorem (Mehlhorn, §): A functional F(f, x), where f G (N N ) k and x eW 
is basic feasible if and only if there exists an oracle Turing machine M with 
oracles for functions f and a basic feasible functional K(f,x) such that M 
computes F(f, x) and the running time T(f, x), with a unit cost for each oracle 
query, satisfies 

(V/)(Vx)(T(/,x)<if(/,x)) 

Theorem (Kapron and Cook, g): A functional F(f,x), where / G (N N ) fe 
and x G N' is basic feasible if and only if there exists an oracle Turing machine 
M with oracles for functions f and a second order polynomial P(\f\, \x\) such 
that M computes F(f, x) and the running time T(f, x), with \ fi{z)\ as the cost 
for an oracle query of fi G / at oracle input value z, satisfies 

(V/)(Vx)(T(/,x)<P(|/|,|x|)) 



In this paper we improve both theorems by combining their best features. 

Definition 8 A functional F(f,x) is computable in polynomial time if there 
exists an oracle Turing machine M with oracles for functions f and a second 
order polynomial P(\f\, |x|) such that M computes F(f, x) and for all /, x, the 
running time T(/,x), obtained by counting each oracle query as a single step 
regardless of the size of the oracle output, satisfies 

T(f,x) <P(|/|,|x|). 



Theorem(lO): A functional F(f,x) is a polynomial time computable func- 



tional if and only if it is a basic feasible functional. 
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1.3 Formal Theories of Second Order Arithmetic 

Theories S3, and R2 which we use in this paper were introduced in || and used 
in P|. S 2 is a second order extension of Buss's 5 2 . Theory S 2 is formulated 
in the language of 5 2 , i.e. <, 0, 1, +, •, \x\, [^x\, #, extended (for easier 
bootstrapping) by the function x \ y producing the number consisting of the 
first, more significant y bits of x i. e. x \ y = [x/2^ x ^ v \ , and by a symbol 
for the application functional Ap(f,x). Consequently, we also extend the usual 
set of open axioms BASIC of theories of bounded arithmetic by adding a few 
axioms for \ (we keep the same notation for the extended set). 

The hierarchy of bounded formulas (i.e. formulas whose all quantifiers are 
bounded) is obtained from the corresponding hierarchies of bounded formulas 
of the first order bounded arithmetic, by allowing the application functional 
Ap(f, x) to appear in the atomic formulas. Bounded quantifiers of the form 3x < 
\t\ and \/x < \t\ are called sharply bounded quantifiers. Bounded formulas form a 
hierarchy at whose bottom level are sharply bounded or Sq formulas, i.e. formulas 
whose all quantifiers are sharply bounded. The complexity of an arbitrary 
bounded formula (e.g., , IlJ) is obtained by counting the alternations of 
bounded quantifiers ignoring sharply bounded ones. 

Theory S 2 is obtained from BASIC by adding either one of the following 
two induction schemas for formulas: 

SJ-PIND A(0, f, y) A (Vx)(j4( \_\x\ , /, y) -> A(x, /, y)) - (Vx)A(x, f, y) 

S^-LIND A(0, f, y) A (Vx)(^(as, /, y) - A(x + 1, /, y)) -» (Vx)AQx\, /, y) 

The proof of the fact that theories (BASIC + S^-PIND) and (BASIC + 
S^-LIND) are equivalent is not only standard, (see Q); but is actually fa- 
cilitated by the presence of the function x \ y in our language and its basic 
properties as axioms of BASIC. We will also mention another sequence of 
formal theories, R 2 , important for the study of parallel computability of func- 
tionals, obtained from (again slightly extended) theory BASIC by adding the 
following induction schema for formulas: 

S^-LPIND A(0, /, y) A (Vx)(A([±x\ , f, y) -> A(x, /, y)) - (Vx)A(\x\, f, y) 

First order theory R\ was studied by Allen (||) and, independently, by Clote 
and Takeuti (0) (TNC in their notation). Our techniques apply equally well 
to parallel computable functionals (a generalization of the class NC; see ||). 

Since we do not have functional substitution in our definition of basic fea- 
sible functionals, theories S 2 have no comprehension axioms at all. The most 
important property of the above theories is that definable function(al)s of these 
theories correspond to important complexity classes of functionals. 

Definition 9 A functional F is S^ 1 definable in the theory S 2 , i > 1, if there 
exists a term t(f,x) and a T,\ formula ^ p(f ,x,y) such that 

S\^Vf,x3\y<t(f,x)* F (f,x,y) 
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and 

(N\ (N N D h V/,af * F (f,x,F(f,x)). 

For i > 1 provably definable functionals of S l 2 are functionals of the 
second order polynomial time hierarchy, Of , a natural extension of the usual 
polynomial time hierarchy for functions Uf . In particular, Sj definable func- 
tionals of the theory S 2 are exactly functionals which form the first level of this 
hierarchy, Df which are just the basic feasible functionals. In order to prove 
this fact we use multiple limited recursion on notation (MLRN) due to Cook 
and Kapron ( || ) , of which we give a very simple proof. Our argument for i = 1 
easily generalizes to S 2 and Df for any i. 

1.4 Proof Techniques 

The main novelty of this paper is in the way how we use logic (in particular 
formal theories) as a tool to derive results of recursion-theoretic nature as well 
as results on Turing computability of functionals. 

It is easy to see that every basic feasible functional F(x, /) is computable 
on a polynomial time oracle Turing machine, as well as that it can be obtained 
using polynomially bounded recursion of polynomial length; this follows imme- 
diately from the fact that every basic feasible functional can be majorized by 
a second order polynomial. The difficult part is to prove that if a functional 
is polynomial time computable or computable from basic feasible functionals 
using polynomially bounded recursion of polynomial length, that then it is a 
basic feasible functional. 

The approach used in the original Cobham's proof for the first order case 
is not applicable here, since it is not possible to code directly a complete in- 
stantaneous description of an oracle Turing machine run or of the sequence of 
intermediate values of a recursive procedure after |i| many steps and then eval- 
uate this functional at a value of \i\ which is greater or equal than the number 
of necessary steps for the computation to terminate. This is because both the 
sizes of complete instantaneous descriptions of an oracle Turing machine run 
(or of the sequence of intermediate values of a recursive procedure) and the 
number of steps necessary for termination of a computation of an oracle Turing 
machine (or of a recursive procedure) are bounded by second order polynomials 
which cannot be majorized by basic feasible functionals. However, definitions 
by recursion on notation require such bounds. 

We eliminate the need for such bounds by showing that functionals com- 
putable on a Turing machine or by a recursive procedure with bounds which 
are second order polynomials are definable in the theory S;j>, and then, using 
entirely conventional proof-theoretic methods, we show that functionals which 
are T,\ definable in the theory S\ are exactly basic feasible functionals. 

In order to show that functionals defined using second order polynomials 
are T,\ definable in the theory S5> we replace bounds involving second order 



8 



polynomials with their suitable representations within our formal theory S 2 . 
This representation consists of a sequence of existentially bounded quantifiers 
prefixing a term bound, and is based on the following Lemma. Without any 
loss of generality, from now on we assume that we have only one input function /. 

Lemma: Let P(|/|, \x\) be a second order polynomial of depth d (see section 
||); then there exists a sequence of terms to, . . . , t& containing only operations 
Ap( f, x), x + y, x ■ y, x#y, \ hx\ and the constant 1 such that for all f and all x 
the following four formulas are true in the standard model: 

(Vu)(|«| < P(|/|, |x|) <-» (3 Zl < t (x)) . . . (3z d < t d -i(f, x,zi,..., 
(u < t d (f,x, zx,..., z d ))) 

Thus, a bound involving a second order polynomial can be replaced by a se- 
quence of existential quantifiers bounded by terms. 

We now work in a formal theory and can use objects defined by formulas with 
higher quantifier complexity as well. Also, instead of having to provide explicit 
definitions with appropriate bounds, we can use induction and give "existential" 
proofs. This greatly facilitates our arguments. 

2 Second order polynomials 

The depth d(P) of a second order polynomial is defined (in to be the 
maximal number of nestings of the application functional; thus, the depth 
of the polynomials not involving the application functional is equal to and 
d(Pi + P 2 ) = d(P! ■ P 2 ) = max{d(P 1 ),d(P 2 )}, while d(f(Px)) = d(Pi) + 1. 

The following simple Lemma can be proved by induction on the complexity 
of the definition of B functionals, using monotonicity of the function Az.|/|(z). 

Lemma 1 (Townsend J7^/J For every functional F(f,x) 6 B there exists a 
second order polynomial P(\f\, \x\) such that for all f and for all x 

\F(f,x)\<P(\f\,\x\). 

In order to handle bounds with second order polynomials using proof-theoretic 
means, we inessentially narrow the class of second order polynomials which we 
will be using. We consider only polynomials which have the property that for 
every m smaller than the depth of the polynomial P and every fj appearing 
in P, there exists exactly one sub-polynomial P* of P of the form fj(P*) of 
depth m; also, for any two subpolynomials of depths d± and d 2 if d\ < d 2 then 
for all / and all x, Pdx(\f\, \x\) < Pd 2 (|/|, |x|). Such polynomials are called 
regular polynomials. Every second order polynomial is majorized by a regular 
second order polynomial. To see this just observe that for every function /, 
the function Az.|/|(z) is a monotone increasing function in z and so we can 
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inductively replace several second order polynomials which are arguments of 
functions \fj\ and which are of the same depth, by their sum. Since we will use 
second order polynomials only to majorize other functionals, we will work only 
with regular second order polynomials. From now on, when we say that P is 
a second order polynomial, we actually mean that P is a regular second order 
polynomial. Also, only for the simplicity of our notation, we will assume that we 
have only one second order variable /. It is easy to see that all our arguments 
easily generalize to the cases involving several second order variables. 

The next lemma will be used to replace bounds which are second order 
polynomials with second order terms involving only feasible functionals. We 
use "witnessing points" to approximate from below the value of a second order 
polynomial. 

Lemma 2 Let P(|/|, |a?|) be a second order polynomial of depth d; then there 
exists a sequence of terms to, ■ ■ ■ , t d containing only operations Ap(f, x), x + y, x- 
y, x#y, L^xJ and the constant 1 such that for all f, all x and all u: 

\u\<P(\f\,\x\) «- (3z 1 <t (x))(3z 2 <t 1 (f,x,z 1 ))... 

(3z d < t d -x(f,x,zi, . . . , < t d (f,x, Zi, . . .,z d )) 

Proof: Simple. 

Definition 10 Let P(\f\, \x\) be a second order polynomial and to(x),. . ., 
td{f, x 7 z%, . . . , Zd) he the sequence of terms as in Lemma (^J); then we call this se- 
quence the sequence of terms associated with the polynomial P(|/|, \x\). We will 
often abbreviate the sequence of these terms as t p , and the corresponding quan- 
tifier prefix (3zi < t (x))(3z 2 < t 1 (f,x,z 1 ))...(3z d < t d -i(f,x,Zi,...,z d -i)) 
as (3z<t~P). 

3 Theories of Second Order Arithmetic 

We now define two second order formal theories of arithmetic S2 and Rg; one 
sort of variables range over the set of natural numbers N; the other range over 
the set of functions of type N — » N. We put no constraints on the growth 
rate of functions. These two theories are then used to characterize type 2 fea- 
sible functionals in the same way how Buss's 5*2 is used to characterize feasible 
functions. 

We will use Buss's results on introducing the polynomial time computable 
functions in S\ (see but due to the presence of functions which can be 
of an arbitrary growth rate, we must do more work. Also, to facilitate the 
bootstrapping of our theories, we will add a few more polynomial time functions 
to the language of our theories which are second order versions of Buss's S\. 
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Definition 11 We will denote by L\ the language consisting of the following 
symbols <, 0, 1, +,-, \x\, , #, \ andAp(f, x). Here x \ y denote^ the y most 
significant bits of x. Ap(f, x) is the only mixed-sort symbol for the application 
functional whose value is f(x) in the standard interpretation. 

We will consider the following induction schemas. 

E^-PIND A(0,a) A {Vx)(A(l±x\,a) A(x)) -» (Vx)A(x,a) 

Sj-LIND A(0, a) A (Vx)(A(x, a) -+A(x+l,a)) -> (Vx)A(\x\,a) 

Yj\ - LPIND A(0,a) A (Vx)(A([±x\,a) -> A{x,a)) -► (Vx)A(\x\,a) 

Yj\ - LLIND A(0,a) A <^x){A{x,a) -> A(x + l,a)) -> (Va:)A(||a;||,a) 

One can easily check that by adding to BASIC the following axioms for x \ y: 
x 1 = 0, x > 1 -> x \ 1 = 1, y < \x\ -> x \ y = and y > \x\ -> 

x \ y = x makes the usual proof (see Buss's 0) of BASIC + (S 4 b -LIND) = 
BASIC + (S^-PIND) simple, and that it goes through also for i = 1 equally 
easily. Theories S l 2 are obtained by adding to such slightly extended BASIC 
either (S^-LIND) or (S^-PIND). Theories R| are obtained by first extending 
the language L\ into the language L 2 d = L\ U {— , Bit(x, y),Lsp(x,y) }; then by 
expanding the axioms of BASIC into BASIC + as described in by adding 
basic properties of these functions and finnaly adding either of the two schemas 
Y?> - LPIND or £f - LLIND. 

We now concentrate on theory (E\ — PIND) as our main tool for studying 
the properties of basic feasible functionals. Let a — f,x. We now want to show 
that a functional is a basic feasible functional if and only if it is definable in 
the theory (E\ - PIND). We start with the easier direction. 

Theorem 3 Every basic feasible functional is definable in the theory 
(Si -PIND). 

Proof: We prove by induction on complexity of definition of a functional 
F that there exists a formula 6j?(a, z\, . . . , z k , y) and sequences of terms 
t%,...,tk, tk+i such that 

(Sj - PIND) h Vcf 3zi < h(a) ...3z k < t k (a, z 1: ..., z k -i) (17) 
3!y < ifc+i(a,fi, ...,2k) F (a, zi,.-- , z k ,y) 

which clearly implies that F is Y>\ definable in (H\ — PIND). The proof pro- 
ceeds by induction on the definition of F G B. If F is obtained by functional 
composition or by expansion the proof is straightforward. 

If F is defined by limited recursion on notation from G, H and K, we use 
Buss's function SqBd(a, b) = (26+l)#(4(2a+l) 2 ) (see Q), which puts an upper 

2 Notice that x \ y is MSP(x, y) in Buss's notation which we here conveniently simplify. 
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bound on codes of sequences of length at most \b\ + 1, consisting of numbers 
< a. Then, assuming that 

F(a,0) = G(a) 

F(a,y) = H(d,y,F(a,^y\)) 

F(a,y) < K(S,y) 

by induction hypothesis there are formulas 8# , Qq, @k an d terms ti ,tj G ,t r , 
1 < < k + 1 such that 

(Sj-P/JVD) h ¥5(32° <t G )3g<t G +1 (d,z G )e G (d,z G ,g), 
{Y,\-PIND) h VdVyVv (3z H < i H )3h < t% +1 (d,y,v,z H )Q H (d,y,v,z H ,h) 
(H\-PIND) h VaVy (3z K <t K )3q < t^ +1 (a,y,z K ) Q K (a,y,z K ,q). 

(18) 

then we prove 

(Sj - FIND) h Va Vy (3z K < t K ) 3t<y3w< SqBd(t£ +1 (d, t, z K ), y) 
3z < w®f(&, y, t, w, z K , z) 

where &f is the conjunction of the following formulas: 

(3z° <i G )3g<t G +1 {d,z G ) (9 G (a,z G ,.g) A 3q < if +1 (a, t, z*) 
6jf(a, 0, z K , g ) A (io)o = min(g, q )) 

(Vi < |»|3«i+i <t* +1 (a,t,z K ) e K (d,y\(i + l),z K ,q l+1 ) 
<t£ +1 (a,t,z K ) A(3z H <i H )3h<tX +1 (d,y\i,(w)^z H ) 
Q H {d, y \ i, (w)i, z H , h) A = mm(/i, g i+ i)) A {(w)\ y \ = z) 

(Vj < |y|3 g <if +1 (a,i,z K ) 6*(a,y f j,z K )(? )) 

The proof is straightforward; the first formula corresponds the initial value of 
the function the second to the recursion on notation and the third to bounding 
and the final value of the computation. 

To prove the converse we need definitions by multiple limited recursion on 
notation MLRN due to Cook and Kapron, of which we give a very simple proof. 

Theorem 4 Let G\, G 2 , H\, H 2 , K\ and K 2 be basic feasible functionals, and 
assume that F\ and F 2 satisfy 

F 1 (0,5) = G^d) 
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F 2 (0,d) = G 2 (d) 

F 1 {u 1 d) = Hi(u,Fi(L|uJ,3),F 2 (L|uJ,5),3) (19) 

F 2 (u,a) = H 2 {u,F 1 {\\u\,d),F2{\_\u\,Q.),a) 

Fi(u,a) < K\(u,d) 

F 2 (u,a) < K 2 (u,a,F 1 (u,d)) 

Then F\ and F 2 are also basic feasible functionals. 

Proof: Let K be defined as follows: 

K(0, d)=o 

K(ud) = { ^(LH'") * 3) <#i(£(Lkl, 
\ u otherwise 

K(u, a) < u 

and let K(u,d) = Ki(K(u 7 d) 7 d). Thus, as it is easily proved by induction on 
u, K(u, d) is the largest of the value of Ki(u \ i, d), for u fixed and < i < \u\. 
Thus, the function 

S(u, d) = SqBd(K(u,d),u) 

has the property that it bounds any sequence a of length |u| satisfying: (Vz < 

M)((a),<^iM^)). 

Consider now functionals F and W defined as follows: 

F(0,d,w) = G 2 (d) 

F(u,d,w) = min{H 2 (u,(w) (luhl) ,F(l±u\,d,w),d), K 2 (u,d,(w)\ u \)} 
and 

w(o,d) = (G 1 (d)) 

W(u,d)=min{W(l±u\,d)~H 1 (u,(W(l±u\,d)) i (20) 

I L 2 u -) I 

F{[\u\,d,w{\_\u\,d)),d), s(u,d)} 

where w^a stands for the sequence w extended at the end with an extra term. 
Then, clearly, 

F(u,a,w) < K 2 (u,d,(w)\ u \) (21) 
W(u,a) < S(u,a) (22) 

which implies that these functionals are both basic feasible. Let now 

= (W(u,a)) lul (23) 
F 2 (u,a) = F(u,d,W{u 7 dj) (24) 
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then one can easily prove by (polynomial) induction on u that these functionals 
satisfy the recursive schema ([l9|). The theorem easily generalizes to n > 2 
functionals, in which case we proceed by induction and assume that the claim 
of the Lemma holds for n — 1. Replace the first functional, Fi, by a variable w, 
representing the code of the computation of a functional W, as we did in the 
case of n — 2; hence the number of the remaining functionals is n — 1. ■ 

We now want to show that every functional of type 2 which is T,\ definable in 
(Sj — FIND) is a basic feasible functional. The proof of this fact is identical 
to the Buss proof of the coresponding result for S 1 ^ and the polynomial time 
computable functions, with one single exception. For convenience, instead of 
the original Buss's proof, we will use Sieg's method of Hcrbrand Analyses from 
[ pL0| . The following Lemma replaces Lemma 1.3.4. from Ell. The rest of the 
proof is identical to the first order case; the fact that we have a two sorted 
language has no impact on the rest of the proof. 

Exactly as in the first order case, we introduce auxiliary theories needed for 
the proof. 

(QF(£>) — PIND) stands for the theory on a language whith a symbol for every 
functional in B. Besides the basic axioms it also has defining recursion equations 
for every such symbol, and induction schema for every open formula. 

(n - Sf) - PIND stands for the theory which extends (QF(S) - PIND) 
with the induction schema for formulas in prenex normal form whose prefix 
has at most n bounded existential quantifiers and no sharply bounded universal 
quantifiers. T,\ formulas without any sharply bounded universal quantifiers are 
called strict H\ formulas. 

Lemma 5 Let A be a set containing only existential formulas ( with the exis- 
tential quantifier bounded or unbounded) . Then if (n — Sy — PIND h A, then 
also (QF(B) - PIND) h A. 

Proof: We proceed by induction on the number of applications of the induc- 
tion rule applied to [n — S^-formulas which can contain at most n existential 
quantifiers (i.e. we do not count instances of induction rule applied to open 
formulas) in the I-normal derivations in (n — X|) — PIND of sets of (n — S^)- 
formulas. (Recall that a derivation is I-normal if all cuts are on induction or 
atomic formulas only.) 

Let the claim of the Lemma hold for derivations with k applications of the 
induction rule. Assume that (n — S^) — PIND h A with an I-normal derivation 
d with k+1 such applications of the induction rule; consider a top-most instance 
of the induction rule applied to a (n — S^-formula ip (i.e. an application of the 
induction rule so that all other applications of the induction rule appearing 
above it in the derivation d are on open formulas): 

r(q), -0(0, a) r(a), ~^ip( \_\bj\ , a), ip(bj,d) 
r(<3), tp(t(a),a) 
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We want to prove that we can reduce this application of (n — S^) — PIND 
to an application of open induction. 
If n = 2, then tp (bi, a) is of the form 

< t (bi, a) 3z 2 < t x (bi, a, z±) 8 (b t , z x , z 2 , a) , 

where bi does not appear in the sequence of variables a, and 9 is open. 

Let di and d 2 be the immediate subderivations; by our assumption, they 
are derivations in the theory (QF(B) — PIND). We first replace the derivation 
d 2 leading to the set T(a), ( \_\bi\ , a) ,ip(bi, a) by a derivation d' 2 of the 
set r(a), a < to (|_5&iJ i A c < ^ , <S, a) — > ^9 (\_\bi\ , a, c, aj , 3z\ < 

to (bi, a) 3z 2 < ti (bi, a, Zi) 6 (bi, z\, z 2 , a) , where a and c are new variables 
not previously used in d. As in the first order case, we can find functional F , 
F , F\ and F\ and derivations d\,d 2 of the sets 

T(a),Fo (a) < t (0, d) A F (a) < h (0, a, F (a)) A 6 (o, F (d), F (d), aj 
and 

T(a),a < t ([\bi\, a) A c < t x (Y\bi\, a, a) -> -.0 a, c, a) , 

F x (bi, a, c, a) < t Q (b t , a) A F x (bi, a, c, a) < t x (bi, a, Fi(b t ,a,c,a)) 
A0{bi, Fx(bi,a,c,d), F x (bi, a, c, a), dj . 

Consider now the formula 

(f)(bi,x, y, a) ee x < t (h, a) A 
y <h (h, a, x) A 6 (bi, x, y, a) ; 

then the above sets are of the form 

r(<S), <p(0, F (d), F (d), a) 

and 

T(a), -.</>( [t|AJ, a, c, a) ,(j)(bi, ^1(^,0,0,0), F 1 (b i ,a,c,d), dj 

respectively. Thus, in order to be able to apply the induction rule for open 
formulas, we must find two functionals F and F such that after substituting the 
free variables a and c with the functionals F(bi, a) and F(bi, a), respectively, in 
the following formulas: 

(j)(o,Fo(d),F (d),d) , -^4>([\bi\,a,c,d) , <j)(b l ,F l (bi,a,c,d),F 1 (b i ,a,c,d),d 
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they become of the form a(0,a), a([^bi\, a) and <r(6,,a) respectively. This 
suggests the following definitions of F and F: 

F(0,d) = F (a) 

F(b h d) = A (6 J ,F(Li6 J J,a),/ , (Li6 J J, a), s) 

< t (&i,S) 

and 

F(0,5) = F (a) 

F(6 4 ,a) = A (6i,f(L5&iJ,£f),F(L|6iJ,5),5) 

< ti (bi,S,F(pi,S)) 

By MLRN such definition is correct. 

Now we can apply the induction rule for open formulas and bounded existen- 
tial introduction rule to get a (QF(B) — PIND) derivation of T(o), ip(t(a), a). 
The claim follows now from the inductive hypothesis. 

Theorem 6 The class of provably total functions of (E* — PIND) is exactly 
the class of the B functionals. 

Proof: 

As in the first order case one first shows that if (Ej — PIND) h (Va)3y < 
t(a)<p(a,y) where <j> is a E^ formula, then for some n, (n — E^) — PIND h 
(Va)(3y < t(a))(f>(a,y) and we can also assume that <f> is a strict E^ formula of 
the language which includes a symbol for every function from B. Then by the 
previous Lemma (QF(B) — PIND) h (Va)(3y < t(d))<p(d,y). Now the claim 
of the Theorem follows from the corfresponding fact about (QF(B) — PIND) 
(whose proof is identical to the first order case). ■ 

The following lemma is a crucial tool in our proofs. 

Lemma 7 Let P(\f\, \x\) be a second order polynomial of depth p and t p = 
if (a?), . . . , tp(f, x, zi, . . . , z p ) the corresponding sequence of terms in variables 
z p = z[ , . . . , z p associated with the polynomial P. Then 

Ri h (V/)(Vf)(3u)(Vzi <t p (x))... (Vz p < t p _ t {f, x,z l7 ...,z p ^)) 

(\tp(f,X,Z!,...,Zp)\ < \u\) 

Thus, despite the fact that second order polynomials are not basic feasible, 
they are provably bounded even in the theory R.3,. Notice that the reason why, 
of course, we cannot conclude that second order polynomials are feasible func- 
tionals from the above provability layes in the fact that formula which defines 
the graph of the functional is not a Ej formula but a conjunction of a Ej and 
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a II* formula: 

((Vzi < to(x)) . . . (Vz p < t^(/,x,zi, . . .,z pi ))(t^(f,x,z 1 , . . . ,z p ) < u)) A 
((3zi < t (x)) . . . (3z d < t dl (f,x,Zi, . . .,z dl ))(t d (f,x, Zi, ...,z d ) = u))) 

Proof: Assume the opposite and fix /, x such that for the formula 
^(v,f,x) ee (3zt < t£(x)) . . . (3z p < t^_ 1 (f,x,z 1 , . . ., V-i)) 

(\tP(f,x, Zll ...,z p )\>v) (25) 

we have (Vu) v E'(|m|, /, x). Fix now an arbitrary v and consider the formula 
v, /, x) ee f i, /, x). Then clearly **(0, v, /, x) holds. Assume now that 
^>*(i,v, f,x) holds; then for some z 1} . . . , z p _ l we have (\tp (/, x, z±, . . . ,z p )\ > 
v \ i). Let Cj = tp (/, x, 2 X , . . . , z ); then u f i < Cf. But then \v f (2i)| < 2|f f 
i| = |(w t i) 2 \ and consequently w f (2i) < 2(v \ i) 2 < 2\c\ 2 = |(c#c) 2 |. Thus, 
(3d < (c#c) 2 )(w r (2») = Ml); since (V«)*(|«|,/,£) we get *(« f (2*),/,*), 
i.e. #*(2i,v, f,x). The above argument shows that for the formula \&* the 
following holds: 

**(0, v, /, 5) A (Vi)(**(i, v, /, x) - (2z, v, /, f)). 

Using the corresponding instance of the S^-LPIND, we get (yi)^*(v,\i\, f.x). 
Thus, $f*(v r f), i.e. x). Consequently, (3zi < i£(a?)j . . . (3z p < 

tp_ 1 (f, x, z±, ... , z p -i))(\tp (/, x, zi, . . . , z p )| > u) holds; since w was an arbitrary 
parameter, we get 

(Vv)(3w)(v = \w\). (26) 

Property (§§) and S^-LPIND imply that S^-PIND and consequently also E£- 
LIND both hold. (In fact, using @ again, we can get that and nJ-IND 

also hold.) We now show that for all k < p the following holds: 

(3u fc )(Vzi <tf (*))... (Va* <Ci(/ l))(*fc (/i^ •! < u k) 

For fc = 0, using (|2q), we find u>o such that ^(x) = \wq\ and then apply 
n^-LIND on y in the formula 

(3z* < \w \)(Vz < y)(tf (f,S,z ) < if (/,x> *)) (28) 

to get (3z* < \w \)(Vz < \w \)(t?(f,x,z ) < tf (/, x, z *)), i.e. (3z * < 
^(x))(Vz < i p (x))(if Cf,f,z ) < t?(f,2,z$)). We now take u = tf (/,x,z *). 
Similarly, assuming that 

(3tt fc )(V2i < to (^)) ■ • • ( Vz fc ^ tk-i{f,x,zi, ■ ■ - ,Zk-i))(tk{f,x,zi, ■■■,Zk)< u k ) 

and taking such that |iOfc| = Uk we prove by induction 

< |u; fe |)(Vz fe+ i < \w k \)(t^ +1 (f,x,z k+1 ) <t^ +1 (f,x,z* k+1 )). 

Taking u = t^ +l (f, x, z% +1 ) we get a contradiction with (p5|). ■ 
We can now prove our main results. 
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Theorem 8 Assume that the functional F(f, x, y) is defined from the function- 
al G(f, x) and H(f, x, z, y) by polynomially bounded recursion on notation with 
the bound Q(\f\, \x\, \y\). Then F(f, x, y) is a basic feasible functional. 

Proof: Just replace second order polynomial bound with an appropriate Y,\ 
formula. Then a simple application of — LI N D proves that such a T,\ formula 
defines a total function. ■ 

Theorem 9 Assume that the functional F(f,x) is defined from the functionals 
G and H by polynomially bounded recursion of polynomial length with bounds 
(Q, P). Then the functional F(f, x) is a basic feasible functionals. 

Proof: Assume that P(|/|, |x|) and Q(|/|, |x|) are two second order polyno- 
mials and let G(f, x) and H(f, x, y, z) be two basic feasible functionals. Let the 
functional F*(f,x,y) satisfies 

F*(/,x,0) = G(/,x) 
F* (/, x,y + l) = H(f, x, F* (/, x, y),y), 
(yy < P(\f\,\x\))(\F*(f,x,y)\ <Q(\f\,\x\)) 
(Vy)(y>P(\f\,\x\) - F*(f,x,y) = F*(f,x,P(\f\,\x\)) 
(V/)(V/*)((V| / <P(|/|,|jf|))(F*(/,f ) tf) = F*(r,£,y)) - 
(\/y)(F*(f,x,y)=F*(f*,x,y))) 

and that the functional F(f,x) is defined by 

P(/,x) = P*(/,x,P(|/|,|x|)). (29) 

We replace the functional F(f, x) by F(f, x) defined as follows. At each recursive 
stage u + 1 of the computation of F* (/, x, u + 1) we also evaluate an approxima- 
tion P*(\f\, \x\, u) of P(\f\, \x\) on the basis of the values f{zi) which were used 
in the computations of F*(f,x,z), for z < u. Thus P*(\f\, \x\,u) = P(\fu\, |x|) 
where fu{zi) — f{ z i) f° r au z % such that f(zi) is used in the computations of 
F*(f, x, z), z < u, and f*(w) — otherwise. 

If u+ 1 is larger than P*(\f\, \x\,u) the recursion is aborted and the value 
of F*(f, x, w) for all w > u is set equal to the value F*(f, x, u). 
Claim: Functional F(f, x) defined above has the same values as F(f, x). 

To prove the claim assume u + 1 is (the first value) larger than P* ( | / 1 , | x \ , u) . 
Then P*(|/^|, \x\,u) = P(|/^|, x|) and so for /* the value of the approximation 
P*i\fu\i is equal to the value of the second order polynomial P{\f^\, |x|). 

However, using the condition 

(Vy)(y>P(\f* u \,\x\) - F*(f*,x,y)=F*(f:,x,P(\f*\,\A)) 

we get that the recursion procedure produces for all y > P(|/^|, x|) the same 
value F*(f*,x,P(\f*\, \x\)). Using the condition 

(V/)(Vr)((V2/<P(|r|,|x|))(P*(/,x,y) = P*(/*,x,y)) -> 
(\Jy)(F*(f,x,y)=F*(f*,x,y))) 
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we get that indeed 

(Vy)(F*(f,x,y)=F*(f:,x,y)) 

This implies that 

(Vy)(y>P*(\f:\,W\) ^ F*(f,x>,y) = F*(f:,x,P*(\f:\,\x\)) 

Since P{\f\, \x\) > P*(|/*|, |x|) we get that 

F*(f,x,P(\f\,\x\)) = F*(f:,x,P*(\f*\Ax\)) 

. However, the computation of F*(f*,x, P*(|/„|, |x|)) clearly satisfies definition 
of the computation of F(f,x), which proves our claim. 

We now formalize in (E\ — PIND) definition of F(f,x). We encode the 
computations for the recursion steps and for approximations P* of the second 
order polynomial P on the basis of values f(zi) used in the previous steps of 
recursion. Using the fact that 

(Vy<P(\f\,\x\))(\F*(f,x,y)\<Q(\f\,\x\)) 

and that (Vy)(y > P(|/|,|x|) -> F*(f,x,y) = F*(f,S,P(\f\,\x\)) we can re- 
place second order polynomial bound Q(\f\, \x\) with a H\ formula with the 
existential prefix 3z < jP with therms t associated with the polynomial Q, 
which defines the graph of F(f, x). 

By Lemma ([?]) it is provable in — PIND) that there exists u such that 
u + 1 is larger than P*(\f\, \x\,u) and so the computation of F(f, x) eventually 
terminates. This implies that F(f, x) is provably total in (Sj — PIND) and 
thus a basic feasible functional. By our claim so is F(f, x). ■ 

We can now improve the Cook Kapron result on machine models. Recall that 
a functional F(f,x) is computable in polynomial time if there exists an oracle 
Turing machine M with oracles for functions / and a second order polynomial 
P(|/|, |x|) such that M computes F(f,x) and for all f,x, the running time 
T(f,x), obtained by counting each oracle query as a single step regardless of 
the size of the oracle output, satisfies 

T(f,x)<P(\f\,\x\). 

Theorem 10 A functional F(f, x) is a polynomial time computable functional 
if and only if it is a basic feasible functional. 

Proof: By an easy inductive argument on the complexity of the definition 
of functionals one can show that every basic feasible functional is polynomial 
time computable (This holds even even if we count every oracle query needed 
to get f(z) as \f(z)\ many steps of computation, as it is proved in see Q). 
Assume that for a second order polynomial P(|/|, \x\) a Turing machine halts in 
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P(|/|, I a; I) many steps. Then at each stage of the computation the working tape 
and the oracle input tape can contain at most P(\f\, \x\) many symbols, and so 
the oracle output tape can contain at most \f\(P (|/|, |x|)) many symbols. It is 
easy to see that this implies that the output of such a machine can be obtained 
by polynomially bounded recursion of polynomial length, and thus it is a basic 
feasible functional. 

■ 
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